About | Support | Discussion Forum | Index | Service Status | ros @ Robotics Stack Exchange

The manipulation pipeline has been released; THIS DEMO PAGE IS NOW DEPRECATED. For current information on the manipulation pipeline, and information about how to execute pick and place tasks using ROS and the PR2 see the pr2_tabletop_manipulation_apps stack page and the pr2_pick_and_place_demos package.

Running the ICRA manipulation demo

Description: This tutorial explains how to run the ICRA manipulation demo.

Keywords: ICRA manipulation demo

Tutorial Level: BEGINNER

Overview

Running the demo implies the following:

• position the robot in a good position relative to the table
• on the robot:

  • bring up the robot

  • start the grasping pipeline

  • start the demo executive, which provides both continuous autonomous operation and step-by-step operation based on keyboard commmands

  • optional: start the joystick controller which allows the demo to be paused between stages to give more detailed explanations to the audience, and can also automatically enable the rviz topics that are relevant to each step in the demo
• on the base station:

  • run rviz using the supplied launch file, which sets up all the right marker topics to provide insight into the demo. Note that you must have a patched version of rviz for this to work.

Starting the Demo

On the robot, run the following in separate terminals:

Robot bring-up:

roscore

roslaunch pr2_bringup pr2.launch

Grasping pipeline:

roslaunch grasping_app_launch grasping_pipeline.launch

Demo executive:

rosrun grasping_app_executive icra_demo_executive.py

Position the robot so that the front edge of the base is flush with the front edge of the table (demo stand), and so that the left edge of the base is flush with the edge of the demo stand. The latter is especially important, as the demo currently does not reason about the edge of the table when placing objects; if you are too far to the left, the robot will drop things on the floor, and if you are too far to the right, there will be no room for the right arm to maneuver around the pole holding up the monitor, and thus it will be forced to do everything with the left arm.

On the basestation, run the following in separate terminals:

rosrun pr2_dashboard pr2_dashboard

roslaunch pr2_teleop_booth rviz_grasping.launch

Running the Demo

The icra_demo_executive provides both continuous and step-by-step operation. In the continuous mode, the robot will move objects from one side of the table to another. In step-by-step mode, you can use keyboard commands to perform individual actions.

You can step through a demo using the step-by-step mode. Another option, explained in the next section, is to run the demo in continuous mode and use the joystick controller to pause it in order to provide more in-depth explanations.

Here is the root menu of the demo executive:

type 'start' to start the autonomous demo
r to control the right arm, l to control the left arm
d to detect objects, dc to detect and take a new static collision map, dca to detect, take a new collision map, and clear attached objects
p to pick up an object, pn to pick up the nearest object on the pick-up side
w to place the object, wo to use the last-resort place right away
h to point the head
s to switch pick-up and put-down sides
t to find the table
rm to reset the collision map, tm to take a new collision map
det to detach the object in the gripper
q to quit
press enter to continue

Continuous mode

Inside the terminal with icra_demo_executive.py running, type 'start' and press enter. The robot will go into continuous mode, and keep running until it encounters an error that it does not know how to recover from.

The keyboard interface allows you to preempt the autonomous demo and go back to a useful menu of keyboard commands by typing 's' and then enter. It will go back after the last service call was completed, which may take awhile. You can also pause it by typing 'p' and enter, although you will probably want to pause through the joystick interface instead.

Always have the run-stop nearby, so that you can hit it if and when the robot does something crazy.

The keyboard interface

The keyboard interface is a good way to get the robot out of stuck situations in a manner more graceful than shutting down motors/manually opening the gripper and putting the robot back in a default state. Particularly useful commands (and these are all printed on the screen for easy reference) include:

• 'w' to place an object the robot is holding back on the table (this should only be done if the arms are both off to the side; if they are not, move them there first. If the robot doesn't remember what the object is, perhaps because the program was just started up, it will give you the option of dropping it semi-blindly.)
• 'r' to control the right arm
• 'l' to control the left arm

Within each arm control menu are the following options:

c to close, o to open the gripper
u to go up, d to go down, rt to go right, lt to go left
ay to go away from the robot, td to go toward the robot
jm to go to arm-away-on-the-side angles using move_arm
jc to go to arm-away-on-the-side angles while trying to constrain the pose
j to go to arm-away-on-the-side angles open-loop using the joint trajectory action
e to exit the arm-control menu for this arm (r)
press enter to continue

Notes:

• 'jm' moves the arm back to the side position using the motion planner, in an attempt not to hit things, then falls back on moving open-loop if the motion planner fails.
• 'j' moves the arm back to the side position without avoiding collisions (but the trajectory is hard-coded so as to be OK in most cases, so it is useful for moving back in a hurry when you're pretty sure there's nothing in the way).
• up/down movement commands move 10 cm at a time, while left/right/toward/away movement commands move 2 cm at a time, without avoiding collisions.

A common useful sequence, if the robot gets stuck with its hand around an object on the table when everything crashes:

Once the demo executive is back up, if the right arm is the stuck one, you can type, in rapid succession, separated by 'enter's (the commands will queue up, with each executing after the last finishes):

• 'r' (right arm control menu)
• 'o' (open the gripper)
• 'u' (move the arm up and away from the object by 10 cm)
• 'jm' (move the arm back to the side while trying not to collide with anything)

The joystick demo controller

In order to run the joystick controller run the following command:

roslaunch pr2_teleop_booth pr2_teleop_manipulation.launch

Then you will use the ps3 joystick controller for the robot to control the manipulation demo. The joystick demo controller is intended to provide useful break points in a demo without changing its internal logic.

At various places inside the code, we have placed break points. Each break point has a sync level associated with it: think of it as a "level of detail" for your explanations. At sync level 0, the entire demo runs continuously. At sync level 1, the demo will pause at a few key points, and wait for you to give the continue signal. At sync level 2, the demo will pause more often, allowing you to discuss one particular sub-module in detail. And so on.

The manipulation demo comes with all the break-points pre-placed in the code. All that's needed is a sync server which allows you to set the sync level and issue "continue" signals. This is the demo joystick controller.

Note that this is optional: if you simply do not start it, the demo will run without pausing at any of the break points.

Joystick operation

If none of the top buttons are pressed the joystick controller is in rviz control mode. For a reference of what the joystick controls do when combined with a patched version of rviz please refer to the documentation for the RVIZ controller mapping in pr2_teleop_booth. The arms and base can also be controlled as described on that page.

The other function the joystick can perform is to change the demo synchronizer level, allowing the presenter to interrupt continuous operation and have a finer fidelity of control over the demo to explain particular concepts. In order to send commands in this mode you must hold down the Demo Synchronizer Mode button shown in the above graphic. When this button is held joystick key presses can be used to control demo synchronization. Note that this mode replaces the Head/Sensor mode in pr2_teleop_booth.

Stepping through the demo using the joystick

If the sync level is non-zero, and the application encounters a break point, it will wait for you to issue the continuation signal. If you look in the console where you started the joystick controller, you will see a message about what break point has been encountered. In general, the mesage will tell you something about the step that the robot is about to take.

Here's an example of a sync request message:

...
[ INFO] [1272564203.224089598]: Sync request (level 1): Using motion planner to move arm to pre-grasp position

You can use this opportunity to tell the audience about what's about to happen, and then issue the continuation signal. Once you emit the signal, the demo will continue until the next sync point:

...
[ INFO] [1272564203.224089598]: Sync request (level 1): Using motion planner to move arm to pre-grasp position
[ INFO] [1272564286.918632293]: Emitting signal
[ INFO] [1272564297.969616563]: Sync request (level 1): Executing interpolated IK path from pre-grasp to grasp

In addition, the same module is used to automatically enable/disable rviz topics to provide the most insight into what is about to happen. You must have started the patched version of rviz using the launch file described earlier in this document.

The actual hierarchy of break points in the manipulation demo is described in detail here: manipulation demo sync points hierarchy.

If you are running the joystick demo controller and the robot appears stuck, there is always the chance it is waiting for a continuation signal. Take a look at the console output. If you can not see it (too far away, hidden by other windows, etc) just decrease the sync level all the way (don't worry, it won't go below 0, so you can press "down" as many times as you want) and that should take care of that. If the robot is still not doing anything, something else is wrong.

Troubleshooting/Recovering from stuck situations

If at any point the robot appears to be stuck, look at the following to see what is wrong:

• rviz
• the joystick interface window
• the dashboard robot status window
• the dashboard console
• the terminal with icra_demo_executive.py
• the terminal with grasping_pipeline.launch
• the terminal with pr2.launch

Semi-common things that can go wrong, and the solutions to them

• Most common failure mode: two or more objects are getting clumped into one giant cluster (one green bounding box will encapsulate more than one object).

  • It is generally a good idea to track rviz and see if this happens
  • Keep in mind that even if objects start up far away from each other, they might end up too close as the robot is moving them around
  • If it happens, separate the objects on the table more. The robot will eventually re-detect and try again to grasp the nearest object.
  • If objects end up close to each other, you might want to surreptitiously nudge an object every now and then to create more breathing room
• The robot keeps trying to pick up thin air:
  • See the above case, it is the most likely cause
• The robot keeps trying and failing to grasp one object:
  • Remove the object. Explain that our algorithms are not perfect to any gawkers, and give the robot an easier toy to play with.
• Robot just waits and does nothing
  • robot might be waiting for a continuation signal from the joystick demo controller.
  • check the console output from the joystick controller to see if that's the case
  • issue a signal, or decrease the sync level all the way to 0
• The icra_demo_executive is telling me that a serious error occurred, and to press 'q' to go back to the keyboard interface:
  • Check the other windows to see why a serious error occurred. This often means that one or more crucial things have died, and may need manual restarting.
  • If it looks like whatever tripped the error has fixed itself, press 'q' and 'enter' to go back to the keyboard interface, use the keyboard interface to put down any objects the robot may be holding, then type 'start' to start up the demo again.
  • Otherwise, you probably want to restart grasping_pipeline.launch and icra_demo_executive.py.
• The robot is moving through its trajectories much faster/more jerky than usual all of a sudden, and possibly spinning its wrist around and around rapidly:
  • Trajectory filters has died. Restart grasping_pipeline.launch and icra_demo_executive.py.
• Things are just frozen:
  • Try restarting just grasping_pipeline.launch and icra_demo_executive.py. If that fails, re-launch the robot.
  • If the robot is still screwed up, and you're not sure what to do, ask for help or reboot the robot entirely.